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NEURO-TRIGGERED  TRAINING  (TRIGGER). 

The  objective  of  the  Trigger  project  is  to  determine  the  feasibility  of  a  method  to  accelerate  the 
learning  of  a  task,  and  optimize  its  performance  by  delivering  stimuli  at  instants  when  prepara¬ 
tory  attentional  networks  are  optimal.  We  plan  to  achieve  this  objective  by  determining  the  pres¬ 
timulus  EEG  patterns  associated  with  a  subject’s  accurate  task  performance  using  neural-network 
pattern  recognition,  and  then  traiiung  the  subject  to  produce  those  patterns  on  a  single-trial  basis. 

According  to  the  plan  for  executing  the  project,  we  are  currently  in  the  process  of  checking-out 
the  basic  Trigger  task  presentation  and  data  collection  system.  Brian  Cutillo,  the  Co-PL  is 
managing  the  day-to-day  conduct  of  the  project.  The  programming  is  being  supervised  by  Jim 
Johnston,  a  biophysicist  who  is  experienced  in  both  FG  programming  and  EEG  feedback  sys¬ 
tems.  We  have  adapted  existing  software  to  record  EEG  and  behavioral  information  on  a  486 
PC,  and  are  redesigning  the  stimulus  presentation  and  data  analysis/display  software  in  a  way 
that  will  allow  easy  modification  of  the  Trigger  system  in  the  course  of  future  development.  For 
efficiency,  a  menu  system  has  been  written  which  is  used  to  set  and  adjust  task  and  feedback 
parameters  during  system  development  and  which  will  be  used  later  when  training  subjects.  The 
first  task  implemented  has  been  the  Bimanual  Visuomotor  Task  (Gevins  et  al.,  1989a),  but  the 


modular  program  design  will  allow  the  system  to  utilize  other  tasks. 

Currently  the  system  is  able  to:  1)  present  the  Bimanual  Visuomotor  Task  with  behavioral  feed¬ 
back  and  end-of-block  behavioral  summaries,  2)  record  9  channels  of  EEC  (right  and  left  hand 
EMG  and  EOG  will  be  added  later),  and  the  output  of  2  finger  pressure  transducers  from  a  sub¬ 
ject  performing  the  task,  and  3)  online  quantify  behavioral  variables  including  response  time, 
pressure,  duration,  accuracy  and  the  adaptive  error  tolerance.  We  have  set  up  a  small  new 
recording  room  for  this  project  and  cleared  it  of  60-Hz  electromagnetic  noise. 

Next  steps  are  to: 

1)  Make  pilot  recordings  from  several  "in  house"  subjects  and  use  the  behavioral  variables  to 
form  data  sets  of  accurate  and  inaccurate  task  trials,  statistically  balanced  for  confoimding  vari¬ 
ables  (especially  response  variables). 

2)  Decide  what  features  to  use  in  classifying  the  accurate  and  inaccurate  trials  using  the  SAM 
neural  network  pattern  recognition  program. 

3)  Program  the  PC  to  extract  these  features  on-line. 

4)  Implement  an  on-line  algorithm  to  detect  lateral  and  vertical  eye  movements  and  blinks,  and 
incorrect  finger  movements  or  hand  EMG  activity,  and  abort  the  trial  if  such  activity  occurs. 

5)  Run  the  system  in  full  neurotrigger  mode. 

When  these  goals  are  achieved,  hopefully  by  the  middle  of  year  3,  we  will  begin  pilot  recordings 
to  fine  mne  the  system  prior  to  a  formal  experiment  We  may  decide  to  conven  the  system  to 
the  newly  developed  "in-hat  preamp"  system.  This  will  require  some  additional  hardware  and 
software  modifications,  but  will  result  in  an  more  flexible,  lower  noise  system. 

REPORT  OF  RESULTS  OF  MEMORY  STUDY 

These  are  the  second  set  of  results  obtained  from  the  data  recorded  fixjm  5  Air  Force  fighter  test 
pilots  several  years  ago.  The  first  paper  was  published  by  the  Journal  of  Electroencephalography 
and  Clinical  Neurophysiology  (Gevins  et  al.,  1990,  Effects  of  prolonged  mental  work  on  func¬ 
tional  brain  topogrophy,  76:339-350).  The  second  report  is  titled  "Neuroelectric  evidence  for 
distributed  processing  in  human  working  memory",  and  will  be  included  in  our  next  scheduled 
report. 

STUDY  OF  LINGUISTIC  AND  GRAPfflC  PROCESSES:  AMPLITUDE  AND  LATENCY  ANALYSIS. 

Data  for  this  smdy  were  previously  analyzed  and  reported  in  AFOSR  Final  Technical  Report, 
"Empirical  Model  of  Human  Higher  Cognitive  Brain  Function"  (March  1990).  This  material  and 
accompanying  graphics  are  now  being  prepared  for  publication,  along  with  some  examples  of  our 
new  method  of  Finite  Element  Model  Deblurring  applied  to  two  of  the  language  subjects  for 
whom  we  obtained  MRI  images  (see  Gevins  et  al.,  1991).  We  are  currently  working  on  the 


challenging  project  of  extending  our  measures  of  functional  association  to  these  cases  of  43- 
channel  data. 

DEVELOPMENT  OF  INTERDEPENDENCY  MEASURES 

Our  ongoing  development  and  validation  studies  of  the  methods  of  measuring  functional  associa¬ 
tions  between  signals  from  widely  distributed  areas  of  the  brain  is  being  supervised  by  Alan 
Gevins  and  Brian  Cutillo.  The  other  personnel  are  working  under  support  from  grants  from 
NINDS,  ONR,  and  NIMH. 

Our  current  effort  is  based  on  detection  and  separation  of  evoked-potential  (EP)  components 
using  a  model  of  linear  stochastic  mixing.  The  analysis  is  currently  being  carried  out  on  a 
single-subject  recordings  of  43-channel  Laplacian  Derivation  language  data,  and  a 
somatosensory-motor  task  using  a  60-channel  chroiucally-implanted  subdural  grid.  Preliminary 
results  suggest  the  existence  of  stable  globally  coherent  spatial  patterns  corresponding  to  the  EP 
components.  Time- lagged  correlations  on  the  EP  components  that  differ  across  stimulus  condi¬ 
tions  may  yield  better  measures  of  interdependency  than  the  untreated  timeseries,  and  may  give  a 
better  indication  of  the  differential  temporal  and  spatial  activation  of  widely  distributed  sites  of 
the  brain  involved  in  the  stages  of  cognitive  processing. 
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Seeing  Through  the  Skuil:  Advanced  EEGs  Use  MRIs  to 
Accurately  Measure  Cortical  Activity  from  the  Scalp 

Alan  Gevins,  Jian  Le,  Paul  Brickett,  Bryan  Reutter,  and  John  Desmond 


Summaiy;  There  is  a  vast  amount  of  untapped  spatial  information  in  scalp-  recorded  EEGs.  Measuring  this  information  requires  use  of  many 
electrodes  and  application  of  spatial  signal  enhancing  procedures  to  reduce  blur  distortion  due  to  transmission  through  the  skull  and  other  tissues. 
Recordings  with  124  electrodes  are  now  routinely  made<  and  spatial  signal  enhancing  techiuques  have  been  developed.  The  most  advanced  of  these 
techniques  uses  udormation  from  a  subject's  MRl  to  correct  blur  distortion,  in  effect  providing  a  measure  of  the  actual  cortical  potential  distribution. 
Examples  of  these  procedures  are  presented,  including  a  validation  from  subdural  recordings  in  an  epileptic  patient.  Examples  of  equivalent  dipole 
modeling  of  the  somatosensory  evoked  potential  are  also  presented  in  which  two  adjacent  fingers  are  clearly  separated.  These  results  demonstrate 
that  EEG^  can  provide  images  of  superficial  cortical  electrical  activity  with  spatial  detail  approaching  that  of  015  PET  scans.  Additionally,  equivalent 
dipole  modeling  with  EEGs  appears  to  have  the  same  degree  of  spatial  resolution  as  that  reported  for  MEGs.  Considering  that  EEG  technology  costs 
ten  to  fifty  times  less  than  other  brain  imaging  modalities,  that  it  is  completely  harmless,  and  that  recordings  can  be  made  in  naturalistic  settings  for 
extended  periods  of  time,  a  greater  investment  in  advancing  EEG  technology  seems  very  desirable. 


Key  words:  High-resolution  EEG;  Evoked  potential;  MRl;  Spatial  signal  enhancement;  Deblurring;  Finite  element  model;  Laplacian  derivation. 


Introduction 

Although  the  EEG  has  been  measured  for  over  60 
years,  and  the  averaged  evoked  potential  for  over  30, 
their  full  potential  as  brain  imaging  technologies  has  not 
yet  been  realized.  This  is  not  due  to  an  inherent  lack  of 
information  in  EEGs,  but  to  a  relative  lack  of  commitment 
of  resources  to  get  at  it.  The  quantity  and  quality  of 
information  obtainable  from  EEGs  is  currently  limited  by 
the  number  of  scalp  recording  sites  and  the  amount  and 
type  of  numerical  computing  applied.  Since  the  former 
is  only  a  matter  of  habit,  cmd  since  computing  has  become 
so  powerful  and  inexpensive,  we  feel  that  it  is  now 
timely  for  the  imaging  capability  of  EEGs  to  make  a  major 
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advance. 

Much  of  the  groundwork  for  this  development  has 
already  been  accomplished,  including  recording  from 
more  than  100  sites  for  improved  spatial  sampling 
(Gevins  et  al.  1990),  precise  measurement  and  registra¬ 
tion  of  electrode  positions  with  3-D  MRl  images  (Gevins 
1989),  means  for  reducing  the  spatial  blm  distortion 
which  occurs  when  potentials  are  conducted  through 
the  skull  (Gevins  1989;  Gevins  et  al.  1990;  Le  and  Gevins 
In  Prep.  a,b),  computation  of  the  "center  of  mass"  of 
cortical  areas  activated  by  sensory  stimulation 
(equivalent  dipoles;  Fender  1987),  extraction  of  spatial 
multivariate  .'eatures  and  classification  of  spatial  brain 
states  using  neural-network  pattern  recognition  techru- 
ques  (Gevins  1980;  Gevins  and  Morgan  1988),  split- 
second  measures  of  functional  cortical  networks  (Gevins 
et  al.  1981, 1989),  development  of  quantitative  norms  for 
clinical  studies  (John  19^,  emd  a  myriad  of  other  techni¬ 
cal  developments  (reviews  in  Lopes  da  Silva  et  al.  1986; 
Duffy  1986;  Gevins  and  Remond  1987;  Basar  1988; 
Pfurtscheller  and  Lopes  da  Silva  1988;  also  see  special 
issues  of  Brain  Topography,  1989,  Vol.  2(1/2),  and  1990 
Vol  3(1)).  Although  a  considerable  effort  has  already 
been  made  in  many  of  these  areas,  an  additional  major 
effort  will  be  required  to  further  refine,  integrate  and 
validate  EEG  spatial  enhancement  methods,  and  then  to 
disseminate  technologies  embodying  them. 

In  considering  whether  it  is  worth  further  developing 
and  modernizing  the  EEG,  it  is  relevant  to  note  that  every 
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bredn  imaging  modality  has  its  relative  strengths  and 
limitations  in  terms  of  spatial  and  temporal  resolution, 
the  nature  of  the  processes  measured,  and  economic  and 
logistical  factors.  For  example,  while  015  PET  is  one  of 
the  imaging  methods  with  the  best  spatial  resolution, 
which  is  approximately  6  to  100  mm  for  modem  PET 
machines  (full  width  at  half  maximum;  M.  Raichle,  per¬ 
sonal  communication;  Mintun  et  al.  1989),  015  PET  also 
has  a  number  of  disadvantages  which  are  not  often  con¬ 
sidered:  1)  the  required  time  sample  of  45-60  seconds  is 
far  too  long  to  measure  split-second  neural  processes  of 
seizure  generation  or  of  cognition;  2)  the  experimental 
designs  for  PET  are  highly  restricted  by  Siifety  limitations 
on  allowable  dosages  of  ionizing  radiation;  and  3)  it  costs 
roughly  five  million  dollars  for  a  PET  facility.  While  not 
comparable  to  PET  in  3-D  resolution  of  many  simul¬ 
taneously  active  areas  throughout  the  neuroaxis,  we 
believe  that  improved  EEG  measures  can  provide  images 
of  superficial  cortical  activity  with  a  spatial  resolution  of 
1-2  square  centimeters,  and  an  unsurpassed  millisecond- 
range  temporal  resolution.  Low  cost  (in  the  one  hundred 
to  two  hundred  thousand  dollar  range),  complete  harm¬ 
lessness,  ability  to  record  for  extended  periods  of  time  in 
a  comfortable  setting,  and  opportunity  to  record  the  same 
subject  many  times  make  EEG  measures  additionally 
attractive. 

124-Channel  EEG  Recordings 

Until  recently  it  was  assumed  by  most  researchers 
that,  due  to  the  smearing  effects  of  volume  conduction, 
the  19  electrodes  of  the  basic  10-20  system  were  sufficient 
for  sampling  the  spatial  information  of  EEG  or  EP  signals 
at  the  scalp.  This  is  clearly  not  the  case,  as  has  been  amply 
demonstrated  (Lehmann  1986;  Wang  et  al.  1989;  Gevins 
1989, 1990).  Figure  1  is  an  example  which  shows  that  the 
EEG  as  normally  recorded  is  spatially  undersampled. 
Isopotential  maps  are  drawn  on  a  scalp  surface 
reconstructed  from  horizontal  MR  images  of  that  subject. 
Data  shown  are  somatosensory  evoked  potentials  to  15- 
Hz  stimulation  of  the  left  middle  and  right  index  fingers, 
for  a  maximum  of  122  channels  and  desampled  to  57, 31 
and  18  channels.  The  spline  interpolation  algorithm 
makes  all  the  maps  visually  appealing,  but  only  in  the 
122-channel  version  is  the  presence  of  two  separate 
peaks  obvious,  corresponding  to  the  two  fingers  stimu¬ 
lated. 

With  the  original  nineteen  electrodes  of  the  10-20  Sys¬ 
tem  (Jasper  1958),  the  typical  distance  between  electrodes 
on  an  average  adult  male  head  is  about  6  cm;  with  124 
electrodes,  the  typical  distance  is  2.25  cm.  This  is  a  good 
improvement  in  sampling  resolution,  but  further  im¬ 
provements  are  possible  since  the  3  dB  point  of  the  point 
spread  function  for  conductance  of  potentials  from  the 


brain  surface  to  the  scalp  averages  about  2.5  cm  (Gevins 
1990).  Thus,  additional  information  could  be  obtained  by 
recording  with  more  electrodes  spaced  closer  together, 
and  then  applying  signal  enhancing  methods  such  as 
those  described  below  to  extract  more  independent  infor¬ 
mation  from  each  electrode.  A  good  goal  for  fuhu-e 
development  is  256  chaimels  which  provides  an  inter¬ 
electrode  distance  of  about  1.6  cm. 

Previous  papers  have  described  our  traditional 
methods  for  recording  EEGs  from  124  scalp  sites  and 
measuring  the  three-dimensional  position  of  each 
electrode  (Gevins  1989;  Gevins  et  al.  1990).  Advanced 
systems  for  very  rapid  electrode  placement  and  position 
measurement  are  currently  being  refined  and  tested  in 
our  laboratory.  The  two  previous  papers  have  also 
described  our  fifth  generation  EEG  analysis  software 
system  which  is  a  UNIX-based,  network-distributed  sys¬ 
tem  written  in  the  C  language.  A  sixth  generation  system 
is  currently  under  development  in  the  C++  language.  It 
is  also  a  network-distributed  system  under  UNIX,  but  it 
is  based  on  the  X-11  network  windowing  standard  and 
incorporates  an  object-oriented  database  technology 
which  will  facilitate  processing,  viewing  and  retrieving 
multichannel  time  series  and  three-dimensional  image 
data. 

MRI  Analysis  and  Modeling  Methods 

Our  approach  to  3-D  anatomical  modeling  and 
visualization  has  been  guided  by  our  need  to  integrate 
anatomical  data  obtained  from  MRIs  with  functional 
data  obtained  from  scalp  and  cortical  EEGs.  Although 
there  are  now  several  commercial  visualization  software 
packages  which  can  generate  3-D  perspective  pictures 
from  MRIs,  we  had  to  develop  our  own  contouring  and 
surface-based  reconstruction  techniques  and  associated 
MR  processing  and  display  methods  as  we  are  more 
concerned  with  3-D  mathematical  modeling  of  the 
anatomical  structures  in  the  data  than  with  visualization 
per  se.  By  contrast,  the  commercial  packages  are  con¬ 
cerned  primarily  with  volume  rendering  (Levoy  1988; 
Levin  et  al.  1989)  or  surface  modeling  for  visualization 
using  discontiguous  "ribbons"  (Heffeman  and  Robb 
1985;  Jack  et  al.  1990;  implemented  by  the  Analyze 
software  package  of  the  Mayo  Foundation)  or  unstruc¬ 
tured  lists  of  triangles  created  by  the  "marching  cubes" 
algorithm  (Lorensen  and  Cline  1987;  implemented  in  the 
apE  software  package  of  the  Ohio  Supercomputer  Cen¬ 
ter). 

We  developed  methods  and  software  to  do  automated 
threshold-based  2-D  contouring  of  individual  MR  im¬ 
ages,  with  optional  manual  editing,  followed  by  the  con¬ 
struction  of  triangular  3-D  surface  elements  between 
contours  in  adjacent  images.  Our  MR  images  have  pic- 
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hire  elements  (pixels)  with  dimensions  that  are  roughly 
1  mm  by  1  mm,  and  typically  the  image  planes  are 
separated  from  one  another  by  3  mm.  Renderings  of 
these  surface  models  are  created  by  using  standard  3-D 
computer  graphics  techniques  (Foley  et  al.  1990).  For 
example,  the  surface  is  "illuminated"  by  a  combination 
of  direct  light,  skylight,  and  diffuse  light,  and  the  amount 
of  light  reflected  from  the  surface  to  an  arbitrary  view¬ 
point  is  calculated.  In  addition  to  generating  these  realis¬ 
tic  surface  renderings,  we  are  able  to  display  data  such  as 
EPs  or  event-related  covariances  (a  measure  of  function¬ 
al  networks  —  Gevins  imd  Bressler  1988)  on  these  sur¬ 
faces.  This  results  in  powerful  images  that  help  to  convey 
the  3-D  complexity  of  anatomical  structure  and  function. 
Previous  papers  have  described  our  EEG-MRI  alignment 
procediu-es  (figure  2),  as  well  as  our  basic  MRI  image 
analysis,  recognition  and  visualization  methods  (Gevins 
et  al.  1990;  Reutter  and  Gevins  In  Prep.). 

EEG  Spatial  Enhancement  Methods 

Electrical  currents  generated  by  sources  in  the  brain 
are  volume  conducted  through  brain,  CSF,  skull  and 
scalp  to  the  recording  electrodes.  Because  of  this,  poten¬ 
tials  due  to  a  localized  source  are  spread  over  a  consider¬ 
able  area  of  scalp  and  the  potential  measured  at  a  scalp 
site  represents  the  summation  of  signals  from  many  sour¬ 
ces  over  much  of  the  brain.  We  have  developed  two 
spatial  enhancement  methods  to  correct  this  blur  distor¬ 
tion;  neither  method  requires  specification  of  an  arbitrary 
source  model  (e.g.,  current  dipoles). 

The  simpler  method  computes  a  very  accurate  es¬ 
timate  of  the  surface  Laplacian  Derivation  (LD),  which 
is  proportional  to  local  normal  current  at  the  scalp.  This 
has  the  advantage  of  eliminating  the  effect  of  the  refer¬ 
ence  electrode  used  for  recording,  and  of  eliminating 
much  of  the  common  activity  due  to  either  the  reference 
electrode  or  volume  conduction  from  distmt  sources. 
The  disadvantages  are  that  the  LD  does  not  produce  valid 
values  at  the  outermost  ring  of  electrodes  and  it  does  not 
correct  for  local  differences  in  skull  thickness  and  con¬ 
duction  properties.  Although  computing  the  LD  at  '"irst 
seems  trivial  (Hjorth  1975),  there  are  in  fact  a  number  of 
subtleties  (Nunez  1989).  The  most  accurate  surface  LD, 
which  we  have  implemented,  uses  the  actual  measured 
electrode  positior\s,  and  estimates  the  LD  over  the  actual 
shape  of  Ae  head  using  a  3-D  spline  algorithm  (Le  and 
Gevins  In  Prep.  a).  Figure  3  shows  the  LD  for  the  same 
data  as  in  figure  1  (lower  right).  A  dramatic  increase  in 
spatial  detail  is  apparent  with  the  LD  as  compared  with 
the  linked-ear-reference. 

A  further  improvement  in  distortion  reduction  is  pos¬ 
sible,  in  principle,  by  using  a  finite  element  model  of  the 
cortex,  CSF,  skull  and  scalp  to  estimate  the  potentials 


which  would  actually  be  recorded  on  the  surface  of  the 
brain.  We  call  our  implementation  of  this  method  Finite 
Element  Model  Deblurring  (FEMDB)  (Gevins  et  al.  1990; 
Le  and  Gevins  In  Prep.  b).  The  price  of  the  improvement 
offered  by  FEMDB  is  that  MRIs  have  to  be  recorded  and 
processed  and  many  more  calculatioirs  have  to  be  per¬ 
formed.  Unlike  other  methods  which  estimate  cortical 
potentials  or  currents  (Nicholas  and  Deloche  1975; 
Freeman  1980;  Hill  et  al.  1988;  Sidman  et  al.  1989),  FEMDB 
is  a  true  "downward  continuation"  method  in  that, 
without  prior  knowledge  or  assumptions  about  the 
generating  sources,  the  cortical  potential  distribution  is 
derived  given  the  scalp  potential  distribution  and  a 
realistic  model  of  the  conducting  volume  between  the 
scalp  and  cortical  surfaces.  In  the  FEMDB,  a  transforma¬ 
tion  matrix  is  constructed  based  on  the  geometry  and 
conductivities  of  the  finite  elements  which  predicts  the 
scalp  potentials  for  any  given  set  of  cortical  potenticils. 
Then  an  efficient  iterative  process  is  used  to  find  the 
cortical  potentials  which  result  in  the  closest  fit  between 
this  forward  solution  and  the  recorded  data.  Simulations 
of  the  method  in  a  three  sphere  model  showed  that  the 
estimated  deblurring  results  are  the  same  as  an  analytical 
computation  of  the  exact  solution  (Le  and  Gevins  In  Prep, 
b).  Experiments  in  which  the  skull  conductivity  con¬ 
stant  was  varied  over  a  50%  range  showed  smooth  and 
well-behaved  effects  on  FEMDB  (Le  and  Gevins  In  Prep, 
b). 

Comparison  of  FEM  Deblurring  With 
Subdural  Grid  Recording 

Initial  results  of  FEMDB  (figure  4)  demonstrate  an 
improvement  in  detail  over  the  Laplacian  Derivation 
(figure  5)  and  good  agreement  between  the  computed 
and  the  actual  cortical  evoked  potentials  (figure  6)  (Le 
and  Gevins  In  Prep.  b).  The  data  are  from  a  patient  with 
pharmacologically-intractable  seizures  for  whom  we 
have  both  scalp  and  subdiural  grid  recordings  of  the 
steady-state  somatosensory  EP  elicited  by  15-Hz  electri¬ 
cal  stimulation  of  the  right  hand. 

Examples  of  FEM  Deblum'ng 

An  example  is  shown  of  deblurring  for  a  normal  male 
subject  who  was  stimulated  with  15  Hz  auditory,  visual 
and  somatosensory  stimuli.  Figure  7  shows  the 
deblurred  steady-state  evoked  potentials  for  stimulation 
of  each  of  three  fingers.  Responses  to  the  left  and  right 
fingers  are  appropriately  lateralized,  and  there  is  an  ob¬ 
vious  difference  between  the  middle  and  index  finger  of 
the  right  hand.  Figure  8  shows  the  15  Hz  auditory 
evoked  potential,  as  well  as  the  visual  response  to 
stimulation  of  the  lower  left  and  right  visual  quadrants. 


128 


Gevins  et  ai. 


Figure  1 .  Steady-state  somatosensory  evoked  potentials  Figure  2.  Electrodes  are  schematically  displayed  as  sttkiII 

elicited  by  15-Hz  stimulation  of  left  middle  and  right  index  purple  cylinders,  at  the  actual  measured  positions,  on  a 

finger  are  shown  for  an  increasing  number  of  channels  3-D  model  of  the  subject's  head  constructed  from  his  MR 
from  18  to  122.  The  potential  distributions  are  shown  images, 
mapped  on  a  reconstructed  scalp  surface  made  from 
that  subject's  MR  images.  Only  the  version  with  1 22  chan¬ 
nels  shows  the  true  topography,  and  allows  clear  visual 
identification  of  both  the  left  and  right-sided  peaks. 


Element  Model  Deblurring  steady-state 
somatosensory  evoked  potentials  elicited  by  15-Hz 
Figure  3.  Potentials  referenced  to  digitally  linked  ears  (left  stimulation  of  the  left  hand  of  an  epileptic  patient  who 

—  same  data  as  figure  1 ,  lower  right)  and  3-D  spline  °  64-electrode  recording  grid  implanted  for  purposes 

Laplacian  Derivation  (right)  compared  for  122-channel,  of  surgical  screening.  The  single  large  peak  in  the  poten- 

steady-state  somatosensory  evoked  potentials  elicited  by  map  (left)  is  considerably  sharpened  in  the  deblurred 

stimulation  of  left  middle  and  right  index  fingers.  The  (fiQht),  which  in  addition  shows  some  polarity  rever- 

Laplacian  Derivation  clearly  isolates  peaks  that  were  sals.  (All  surfaces  were  reconstructed  from  horizontal  MR 
merged  together  in  the  potentials.  images  obtained  prior  to  surgery,  one  of  which  is  shown  in 

'  part  between  the  scalp  and  cortex  on  right.) 
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Figure  6.  Original  scalp  potentials  (left  —  same  data  as  Figure  6.  Comparison  of  deblurred  evoked  Fxjtentials  (left 

figure  4,  left)  and  3-D  spline  Laplacian  Derivation  (right).  —  same  data  as  figure  4,  right)  with  the  evoked  p^  ^tentials 

The  Laplacian  Derivation  also  is  spatially  sharpened  com-  actually  recorded  from  the  subdural  grid  (right)  is  shown, 

pared  with  the  original  potential  data,  but  is  less  detailed  The  area  covered  by  the  grid  shows  a  single  large  peak 

than  the  deblurred  data  shown  in  figure  4,  right.  similar  to  that  in  the  deblurred  data. 


Deblurred  evoked  potentials  for  the  visual  stimuli  show 
maxima  located  near  the  occipital  pole,  which  is  the 
location  of  the  most  likely  source,  striate  cortex  (area  17). 
The  auditory  deblurred  evoked  potentials  are  less  clearly 
localized,  as  would  be  expected  for  this  more  difficult 
case  and  the  limited  accuracy  of  our  initial  FEM  model, 
but  are  more  consistent  with  bilateral  temporal-lobe 


sources  than  the  raw  scalp  potentials  would  indicate. 

Equivalent-Dipole  Source  Localization  For 
Somatic  and  Visual  Stimuli 

Single  equivalent  dipole  modeling  of  the  head  was 
performed  for  each  of  the  three  15  Hz  somatosensory 


Figure  7.  Deblurred  steady-state  evoked  potentials 
elicited  by  1 5-Hz  stimulation  of  left  middle  and  right  middle 
and  index  fingers  of  a  normal  subject  (ms3).  All  three 
cases  show  the  expected  contralateral  maximum  in  ac¬ 
tivity,  while  the  response  to  right  index  finger  stimulation 
differs  markedly  from  that  for  the  right  middle  finger. 


Figure  8.  Deblurred  steady-state  evoked  potentials 
elicited  by  15-Hz  auditory  and  visual  stimulation.  Rearview 
of  the  head  shows  maximal  activity  at  the  occipital  pole 
for  the  parafoveal  right  and  left  visual  quadrant  stimula¬ 
tion.  The  auditory  response  is  less  clear,  but  is  greater  over 
the  right  hemisphere. 
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Figure  9.  Single  equivalent  dipole  modeling  for  each  of 
the  three  15-Hz  somatosensory  stimulus  conditions  shown 
in  figure  7.  Dipoles  ore  shown  with  respect  to  the  scalp 
surface  and  o  cubic  brain  model  constructed  from  the 
subject's  MR  images.  Each  dipole  is  represented  as  a 
disk  with  its  center  on  the  dipole's  location,  and  with  a 
cone  pointing  in  the  dipole's  direction.  Dipole  color  indi¬ 
cates  stimulus  as  follows:  left  middle  finger  (blue),  right 
middle  (red),  and  right  forefinger  (yellow).  Each  dipole 
appears  in  the  contralateral  hemisphere,  and  the  dipole 
for  the  forefinger  is  located  slightly  more  lateral  than  that 
for  the  middle  finger,  consistent  with  the  known  locations 
of  the  somatosensory  projection  areas. 


stimulus  conditions  shown  in  figure  7.  Figure  9  shows 
the  dipoles  with  respect  to  the  scalp  surface  and  a  cubic 
brain  model  constructed  from  the  subject's  MRI  (see 
Gevins  et  al.  1990).  Each  dipole  is  represented  as  a  disk 
with  its  center  on  the  dipole's  location,  and  with  a  cone 
pointing  in  the  dipole's  direction.  Dipole  color  indicates 
stimulus  as  follows:  left  middle  finger  (blue),  right  mid¬ 
dle  (red),  and  right  forefinger  (yellow).  The  goodness  of 
fit  was  85,  96  and  97%  for  right  forefinger,  right  middle 
finger,  and  left  middle  finger,  respectively.  Each  dipole 
appears  in  the  contralateral  hemisphere,  and  the  dipole 
for  the  forefinger  is  located  slightly  more  lateral  than  that 
for  the  middle  finger,  consistent  with  the  known  loca¬ 
tions  of  the  sensory  projection  areas  and  other 
physiological  source  localization  results  (Okada  et  al. 
1984;  Wood  et  al.  1985;  Luders  et  al.  1986).  Although 
dipole  models  are  not  physiologically  realistic,  they  are 
nonetheless  useful  for  locating  tne  center  of  mass  of 
primary  sensory  cortex.  Although  some  proponents  of 
MEG  technology  have  maintained  otherwise,  equivalent 
dipole  modeling  performed  with  EEG  appears  to  be  no 
less  accurate  than  that  performed  with  MEG.  This  same 


conclusion  has  recently  been  demonstrated  via  direct 
comparison  of  MEG  and  EEG  localization  for  dipoles 
generated  by  stimulation  of  electrodes  implanted  in 
epileptic  patients  (Cohen  et  al.  1990). 

Conclusion 

Although  hundreds  of  millions  of  dollars  have  been 
invested  in  developing  other  forms  of  functional  brain 
imaging  including  PET,  SPECT,  MRSI,  and  MEG,  the 
EEG  has  been  relatively  overlooked  by  the  scientific, 
medical  and  business  community  at  large.  'This  is  unfor¬ 
tunate  since  it  is  clear  that  EEGs,  coupled  with  MRIs,  are 
capable  of  providing  images  of  superficial  cortical  electri¬ 
cal  activity  with  very  high  temporal  resolution  and  spa¬ 
tial  detail  approaching  that  of  015  PET  scans.  EEGs  are 
inherently  a  low  cost  technology.  'They  require  neither 
bulky,  expensive  sensor  technologies,  nor  ionizing 
radiation.  We  sincerely  hope  that  EEG  will  receive  the 
attention  and  subsequent  large-scale  development  that  it 
merits. 
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INTRODUCTION 


Among  the  techniques  for  studying  brain-behavior  relationships  in  humans,  scalp- 
recorded  neural  potentials  have  been  used  widely  for  over  50  years.  With  the  technologi¬ 
cal  advances  of  the  1970s  and  1980s,  new  recording  and  analysis  tools  have  been 
developed  for  obtaining  increasingly  specific  information  about  the  spatial  and  temporal 
features  of  neurocognitive  processes.  Such  advances  have  made  possible  neurodectric 
recordings  with  many  charnels,  well  beyond  the  original  19  channels  of  the  interna¬ 
tional  10-20  system  proposed  by  Jasper'  in  the  19S0s.  These  tools  have  also  provided 
improved  signal  processing  and  means  of  correlating  neuroelectric  measures  with  aiu- 
tomical  information  from  magnetic  resonance  images.  We  will  focus  on  these  tools  in 
the  first  part  of  this  chapter;  in  the  second  part,  we  will  describe  how  we  have  been 
applying  the  tools  to  study  the  spUt-second  components  of  higher  cognitive  functions. 


TOOLS  OF  THE  TRADE 


Improved  ^atial  Sampling 


* 

Electrode  Arrays  with  125  Channels 


One  of  the  requirements  for  extracting  detailed  information  about  cognitive  pro¬ 
cesses  from  the  s^p-recorded  EEO  is  to  have  adequate  spatial  sampling.  The  19 
channels  customarily  employed  in  clinical  recordings  provide  an  interelectrode  distance 
of  about  6  cm.  Wl^e  this  is  sufBcient  for  detecting  signs  of  gross  pathtfiogy,  it  is 
obviously  insufficient  for  resolving  functional  differences  within  small  cortical  regions. 
To  improve  spatial  resolution,  we  have  been  making  S9-chaimel  recordings  for  the  past 
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several  years.  This  provides  an  interelectrode  distance  of  about  3.5  cm  on  a  typical 
adult  head,  which  is  still  not  good  enough.  To  improve  sampling,  we  recently  developed 
a  12S-channel  recording  system  that  provides  an  interelectrode  distance  of  about  2.25 
cm.  Subjects  wear  a  stretchable  E£G  recording  cap,  with  electrodes  placed  on  the  cap 
according  to  an  expanded  version  of  the  standard  international  10-20  system.’  Prior  to 
each  EEG  recording  session,  the  3-D  position  of  each  electrode  on  the  individual 
subject’s  head  is  measured  precisely  with  a  commercial  3-D  digitizer  (Pig.  1).  To  date 
12  full-scale  12S-channel  recordings  have  been  made  from  subjects  receiving  visual, 
auditory,  and  somatic  stimuli. 


Registration  of  Scalp-Electrode  Positions  with  Underlying  Anatomical  Structures 


To  visualize  the  brain  areas  underlying  the  scalp  electrodes,  a  procedure  is  needed 
for  aligning  scalp-electrode  positions  and  underlying  anatomical  structures.  This  first 
requires  producing  an  accurate  anatomical  representation  of  a  subject’s  brain. 

Distortion  Correction  of  Magnetic  Resonance  Images.  While  magnetic  resonance 
(MR)  images  are  invalu^le  because  of  the  anatomical  differentiation  they  provide, 
they  contain  inherent  distortion  that,  if  not  corrected,  may  cause  quantitative  measure¬ 
ments  of  position,  length,  area,  and  volume  to  be  erroneous.  The  distortion  can  exceed 
10%  and  commonly  arises  from  calibration  errors  and  inhomogeneities  of  the  magnetic 
field  gradients.  We  have  been  working  on  correcting  this  problem  using  both  phantom 
calibration  data  and  data  recorded  from  human  sub^ts  wearing  constructed  helmets 
with  spherical  fiducial  markers  that  are  easily  visual!^  on  MR  images.  It  is  necessary 
to  correct  both  image  intensity  and  image  position. 

To  test  our  methods,  scans  from  a  Diasonics  MTS  MR  system  were  made  on  three 
subjects.  The  maximum  total  distortion  measured  on  this  machine  was  8%.  Vsriation 
between  images  with  TR  =  600  msec  and  TE  =  20  msec  and  images  with  TR  =  2000 
msec  and  TE  =  35  and  70  msec  were  found  to  be  less  than  2%.  We  compared  the 
location  of  the  spherical  fiducial  markers  in  coronal,  sagittal,  and  axial  images,  and 
various  inuige  transforms  were  then  used  to  bring  the  measured  points  into  alignment. 
We  found  that  by  computing  a  separate  scale  factor  for  each  dirwtion  combined  with 
a  translation  and  rotation,  two  sets  of  images  could  be  brought  into  reasonably  close 
alignment.  An  example  of  a  set  of  Diasonics  MRIs  before  correction  is  shown  in 
Figure  2A.  It  is  clear  that  the  anatomical  positions  corresponding  to  the  scalp  and 
cranium  do  not  line  up.  The  coronal  sections  shown  in  blue  are  shifted  to  the  left  of 
their  correct  position.  The  sagittal  sections  shown  in  reddish  brown  are  "stretched”  by  * 
approximately  17%  in  the  anterior/posterior  direction  compared  to  the  horizontal  or 
transaxial  sections  which  are  shown  in  dark  green.  The  same  images  after  correcting 
for  distortion  are  shown  in  Figure  2B. 

Alignment  of  EEG  Electrode  Positions  with  MR  Surface  Reconstructions.  A  linear 
transformation  is  calculated  to  superimpose  electrode  positions  and  the  scalp-surface 
contours  obtained  from  MR  images.  This  transformation  is  initially  determined  by 
visually  adjusting  a  graphical  display  of  the  electrodes  and  scalp  contours.  An  optimal 
transformation  is  then  calculated  by  a  program  that  adjusts  each  parameter  in  the 
transformation  until  the  average  distance  between  all  electrode  positions  and  the  closest 
scalp  point  is  minimized  (Fig.  3).  The  aligned  surface  model  can  also  be  superimposed 
onto  composite  MR  images  showing  various  orientations  (Fig.  4).  Additionally,  sur¬ 
faces  may  be  constructed  by  manually  tracing  other  structures  on  each  MR  image  and 
then  calculating  the  polygonal  surface  which  fits  these  contours  (Fig.  S). 
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FIGURE  1.  Digitization  of  scalp  electrode  positions.  The  subject  is  resting  in  a  headrest  designed 
to  mininaize  head  movement.  The  technician  touches  the  stylus  (which  contains  electromagnetic 
field  sensors  for  x,  y,  and  z  axes)  to  each  of  the  electrodes  in  turn.  The  3-D  coordinates  of  each 
electrode  position  are  transmitted  to  the  data  collection  and  analysis  computer. 
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FIGURE  2.  (A)  Composite  of  distorted  MR  images  in  horizontal,  sagittal,  and  coronal  orienta¬ 
tions  as  originally  recorded.  The  location  of  the  scalp  is  not  consistent  for  different  orientations. 
(B)  Composite  of  the  same  images  after  transformation  to  correct  for  distortion.  The  scalp  surface 
now  appears  at  the  same  location  in  all  orientations. 
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FIGURE  3.  Rough  sagittal  view  of 
128-electrode  monuge  constructed 
from  horizontal  MR  images  and  po¬ 
sitioned  on  the  scalp  surface.  Also 
shown  is  the  coordinate  system  with 
the  origin  located  halfway  between 
the  T3  and  T4  temporal  electrodes. 
Some  electrodes  are  not  properly 
aligned  with  the  scalp  surface  due  to 
a  mechanical  problem  that  has  been 
corrected. 


FIGURE  4.  Rough  surface  recon¬ 
struction  of  the  scalp  from  hori¬ 
zontal  MR  slices  with  a  cutaway  to 
show  horizontal,  sagittal,  and  coro¬ 
nal  MR  images  of  the  left  frontal 
lobe.  In  this  early  reconstruction, 
the  sagittal  and  horizontal  slices  are 
not  entirely  aligned,  causing  the  sa¬ 
gittal  section  to  protrude  from  the 
surface. 


FIGURE  5.  Reconstruction  of  the 
right  hemisphere  of  the  brain  and 
spinal  cord  viewed  from  the  (A) 
right  and  (B)  anteriorly. 
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Reduction  of  Brain  Potential  Blur  Distortion 


Laplacian  Derivation.  Neuroelectric  signals  recorded  at  the  scalp  are  principally 
distorted  by  transmission  through  the  low-conductance  skull.  This  distortion  manifests 
as  a  spatial  low-pass  filtering  which  causes  the  potential  distribution  at  the  scalp  to 
appear  blurred  or  out  of  focus.  There  are  a  number  of  methods  for  reducing  this 
distortion,  among  which  the  spatial  Laplacian  operator  is  perhaps  the  simplest  and 
most  effective.  This  method,  which  is  oten  referred  to  as  the  Laplacian  derivation,  is 
derived  by  computing  the  second  derivative  in  space  of  the  potential  field  at  each 
electrode.  This  converts  the  potential  into  a  quantity  proportional  to  the  current 
entering  and  exiting  the  scalp  at  each  electrode  site,  and  eliminates  the  effect  of  the 
reference  electrode  used  during  recording.  An  approximation  to  the  Laplacian  deriva¬ 
tion,  introduced  by  Hjorth^-^  assumes  that  electr^es  are  equidistant  and  at  right  angles 
to  each  other.  Although  this  approximation  is  fairly  good  for  some  electrode  positions 
such  as  midline  central  (Cz),  it  is  less  accurate  for  others  such  as  midtemporal  (TS). 
We  have  been  using  a  more  accurate  estimate  of  the  Laplacian  derivation  that  is 
based  on  projecting  the  measured  electrode  positions  onto  a  two-dimensional  surface. 
Although  this  produces  a  dramatic  improvement  in  topographic  detail,  some  problems 
remain  because  of  the  assumptions  that  surrounding  electrodes  used  to  estimate  the 
Laplacian  of  an  electrode  are  near  that  electrode  and  that  the  current  gradient  is 
uniform  over  the  region  encompassed  by  the  surrounding  electrodes.  Furthermore,  it  is 
not  possible  to  estimate  the  Laplacian  at  peripheral  electrodes  because  the  surrounding 
electrodes  are  incomplete. 

Spatial  Deconvolution  Using  Spherical  Head  Model  By  modeling  the  tissues  be¬ 
tween  brain  and  scalp  as  surfaces  with  different  thicknesses  and  resistances,  we  have 
performed  a  deblurring  operation  that,  in  principle,  makes  the  potential  appear  as  if  it 
were  recorded  just  above  the  level  of  the  brain  surface,’  without  assumptions  about  the 
actual  (cortical  or  subcortical)  source  locations.  The  deblurring  operation,  however, 
requires  detailed  modeling  of  the  tissues  which,  when  the  exact  shape  of  the  head  is 
taken  into  account,  is  a  great  deal  of  work.  The  operation  is  even  further  complicated 
by  the  fact  that  a  solution  to  calculating  the  local  resistance  of  the  skull  precisely  does 
not  yet  exist.  With  the  conduction  of  potentials  from  a  localized  source  spread  over  a 
considerable  area  of  scalp,  the  summation  of  signals  at  any  given  scalp  site  may  reflect 
many  sources  over  much  of  the  brain.  In  the  context  of  a  four-shell  spherical  head 
model,  we  have  estimated  the  amount  of  spread— the  “point  spread”— for  a  radial 
equivalent  dipole  source  in  the  cortex  to  be  about  2.S  cm.  If  the  conductance  of  the 
skull  is  known,  a  deblurring  operation  using  a  model-based  deconvolution  can,  in 
principle,  achieve  a  better  signed  enhancement  than  a  Laplacian  derivation  when  the 
^stance  between  electrodes  is  less  than  approximately  the  point  spread  distance  of 
2.3  cm. 

Finite  Element  Method.  Another  method  of  increasing  spatial  resolution  for 
those  cases  for  which  the  source  generators  can  be  modeled  as  current  dipoles,  and  for 
which  MR  data  is  available,  is  the  finite  element  method  (FEM).  The  entire  volume  of 
the  head,  as  found  in  MR  images,  is  broken  up  into  many  small  elements  representing 
various  tissues;  the  scalp,  skull,  and  brain.  By  assigning  each  element  a  conductivity 
constant  (obtained  from  textbook  values)  and  one  for  a  known  source,  it  is  possible  to 
calculate  the  potential  at  each  vertex  of  all  the  finite  elements  using  Maxwell’s  equations. 
Because  the  number  of  vertices  is  approximately  10,000,  an  efficient  algorithm  is 
necessary  to  make  this  practical  on  a  sm^l  computer.  Using  a  SUN  Sparc-1  workstation 
rated  at  about  12  MIPS,  the  initial  matrix  decomposition  based  on  a  set  of  finite 
elements  takes  about  90  min,  while  the  potential  computation  for  each  source  takes 
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6  min.  If  a  practical  method  can  be  developed  for  estimating  local  skull  conductance, 
the  FEM  deblurring  method  has  the  capability  of  producing  highly  enhanced  represen¬ 
tations  of  the  current  distribution  on  the  exposed  surface  of  the  cortex. 


Detecting  Artifacts 


The  usual  practice  in  evoked  potential  studies  of  cognition  is  to  reject  automatically 
artifacted  trials  in  which  the  voltage  of  the  eye-movement  measurement  channels 
exceeds  a  fixed  threshold.‘  While  this  procedure  catches  large  contaminants,  it  entirely 
misses  small  ones.  This  can  lead  to  a  spurious  result  if  there  are  small,  but  consistent 
saccades  or  microblinks  approximately  time-locked  to  stimulus  presentation.  Although 
we  also  use  an  on-line  artifact  detection  procedure  to  flag  automatically  portions  of 
trials  and  individual  electrodes  that  have  unusually  high  or  low  amplitude,  all  data  are 
examined  visually  on  a  graphics  terminal  to  confirm  and  improve  the  computer’s 
detections  as  needed.  In  our  studies  with  clinicaUy  healthy,  young  adult  subjects,  there 
is  about  10%  data  attrition  due  to  artifacts. 


Data  Set  Formation:  Controlling  for  Spurious  Sources  of  Variance 


After  the  data  have  been  cleared  of  instrumental  and  subject-related  artifacts,  data 
sets  are  usually  formed  in  pairs  to  test  specific  hypotheses.  In  forming  these  data  sets, 
it  is  imperative  that  the  major  difference  between  two  sets  be  related  to  the  hypothesis 
being  tested.  It  is,  of  course,  standard  practice  to  try  to  eliminate  spurious  differences 
by  careful  experimental  design,  but  there  is  always  the  chance  that  some  remaining 
factors  differ  between  sets.  These  uncontrolled  factors  can  include  small  residual  eye- 
movement  contaminants,  arousal  level,  and  response  movement  parameters  (e.g.,  force 
or  reaction  time),  all  of  which  are  known  to  affect  neuroelectric  signals. 

To  ascertain  that  the  major  source  of  variance  b  actually  related  to  the  hypothesis, 
the  two  sets  of  artifact-free  trials  are  submitted,  usually  on  a  subject-by-subject  basis, 
to  an  interactive  program  that  displays  the  means,  t  tests,  and  histogram  distributions 
of  up  to  SO  behavioral  and  physiological  event  variables.  These  include  stimulus  parame¬ 
ters,  reaction  time,  movement  magnitude  and  duration,  error,  EEC  arousal  index, 
eye-movement,  muscle  potential  indices,  and  so  on.  The  data  sets  are  inspected  for 
significant  differences  in  variables  which  are  not  related  to  the  hypothesis,  and  outliers 
are  discarded.  As  an  example,  an  unintentional  difference  between  experimental  condi¬ 
tions  in  response  force  may  be  present.  In  the  data  set  with  the  larger  response  force, 
the  associated  movement-relat^  potentials  could  overlap  the  P300  evoked  potential 
peak  causing  a  spurious  between-condition  difference  in  P300  amplitude.  After  careful 
balancing  of  the  data  sets  for  such  movement  parameters,  valid  assessments  about  P300 
peak  effects  may  be  made.  In  balancing  our  data  we  are  careful  not  to  truncate  the 
histogram  distribution  severely.  The  unrelated  variables  are  reduced  to  a  between- 
condition  alpha  significance  of  0.2,  or  if  this  is  not  possible  without  seriously  affecting 
the  distribution,  to  just  over  O.OS.  The  net  effect  of  these  procedures  is  the  certainty 
that  when  a  neuroelectric  difference  between  experiment^  conditions  is  found,  the 
difference  actually  relates  to  the  hypothesis  under  consideration. 
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Neurocognitive  Pattern  Analysis 


We  have  been  using  the  term  “neurocognitive  pattern  analysis”  (NCP  analysis)  to 
refer  to  our  procedures  for  extracting  task-related  spatiotemporal  patterns  from  the 
unrelated  background  activity  of  the  brain.  In  the  first  of  three  generations  of  NCP 
analysis,  we  measured  background  EEC  spectral  intensities  while  people  performed 
complex  tasks,  such  as  arithmetic  problems  lasting  up  to  one  minute.  These  patterns 
had  suflScient  specificity  to  identify  the  type  of  task,^-'  but  when  the  tasks  were  controlled 
for  stimulus-,  response-,  and  performance-related  factors,  they  had  identical,  spatially 
diffuse  EEC  spectral  scalp  distributions.^  This  study  suggested  that  complex  tasks 
involving  a  variety  of  sensory,  cognitive,  and  motoric  processes  activate  large,  wide¬ 
spread  areas  of  cortex  to  a  degree  proportional  to  the  subject’s  effort.  It  also  strongly 
suggested  that  most  studies  of  EEG  correlates  of  cognitive  activities,  including  those 
of  hemispheric  lateralization,  may  have  confounded  electrical  activity  related  to  limb 
and  eye  movements,  stimulus  properties,  and  task  difiSculty  with  those  of  mental  activity 
per  se. 

In  the  second  generation  of  NCP  analysis,  we  measured  cross-correlations  between 
electrodes  recorded  during  performance  of  simple  visuomotor  judgment  tasks.’  From 
this  experiment,  in  which  rapidly  shifting  focal  patterns  were  extracted  from  two  similar 
spatiai  Usks,  it  was  clear  that  a  split-second  temporal  resolution  is  imperative  for 
isolating  the  rapidly  shifting  neurocognitive  processes  associated  with  successive  infor¬ 
mation  processing  stages. 

In  the  third  generation,  we  extended  our  methods  to  include  event-related  covari¬ 
ances  (ERCs).  The  ERC  approach  is  based  on  the  hypothesis  that  when  regions  of  the 
brain  are  ftmctionally  related,  their  event-related  potential  (ERP,  another  name  for 
evoked  potential)  components  are  related  in  shape  and  in  time.'°  The  idea  is  that  the 
ERP  waveform  delineates  the  time  course  of  event-related  mass  activity  of  a  neural 
population,  so  that  if  two  populations  are  functionally  related,  their  ERPs  should  hne 
up  in  time,  perhaps  with  some  delay.  If  so  (and  if  the  relationships  are  linear  as  they 
often  appear  to  be),  this  could  be  measured  by  the  lagged  covariance  between  the  ERPs, 
or  portions  of  the  ERPs,  from  different  regions  (Fic.  6).  This  is  the  event-related 
covariance  method. 

The  procedures  that  are  followed  for  ERC  analysis  are  described  here.  Procedures 
1  -4  have  been  discussed  in  detail  above. 


1.  A  sufficient  amount  of  data  are  recorded  using  as  many  electrodes  as  possible. 

2.  Data  with  artifact  contamination  are  removed. 

3.  Pairs  of  conditions  to  be  compared  are  selected,  and  trials  with  extreme  values 
of  behavioral  variables  are  eliminated. 

4.  The  Laplacian  operator  is  applied  to  the  potential  distribution  of  each  non¬ 
peripheral  scalp-electrode  location. 

5.  Analysis  intervals  and  digital  filter  characteristics  are  determined.  The  analysis 
intervals  are  usually  either  centered  on  an  ERP  peak,  or  are  positioned  just 
before  or  after  a  stimulus  or  response. 

6.  Enhanced,  filtered,  and  decimated  averaged  Laplacian  ERPs  for  each  condition 
are  computed.  (In  an  optional  procedure,  used  when  the  signal-to-noise  ratio 
is  very  low,  a  statistical  procedure  is  used  to  identify  trials  with  measurable 
event-related  signals,  and  averages  are  formed  only  ftx>m  those  trials.") 

7.  Multilag  cross-covariance  functions  are  computed  between  all  pairwise  channel 
combinations  of  these  averaged  ERPs  in  each  selected  analysis  window.  The 
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magnitude  of  the  maximum  value  of  the  cross-covariance  function  and  its  lag 
time  are  the  features  used  to  characterize  the  ERC.  The  covariance  analysis 
interval  is  the  width  of  one  period  of  the  band-center  frequency  of  each  filter. 
Down-sampling  factors  are  determined  by  the  20  dB  rejection  point,  and  the 
covariance  function  is  computed  up  to  a  lag  time  of  one-half  period  of  the  high 
frequency  for  each  band.  For  example,  we  often  use  a  filter  with  3  dB  cutoffs 
at  4  and  7  Hz,  and  with  20  dB  attenuation  at  l.S  and  9.S  Hz.  The  filtered 
time  series  are  decimated  from  128  to  21  Hz  for  each  covariance  calculation. 
Covariance  is  estimated  over  a  187-msec  window,  which  corresponds  to  one 
period  of  a  S.S-Hz  sinusbid.  Each  window  is  lagged  by  up  to  8  lags  at  the 
original  undecimated  sampling  rate,  i.e.,  one  hundred-twenty-eighth  of  a  second 
per  lag. 


FIGURE  6.  Schematic  diagram  showing  the  relationship  of  an  event-related  covariance  (ERC) 
line  on  a  top  view  of  a  model  head  (.left)  to  the  theta-band-fUtered,  averaged  event-related  Laplacian 
derivation  waveforms  {right).  ERCs  were  computed  over  the  indicated  187-msec  analysis  interval 
from  the  aPz  and  aCz  electrode  sites.  The  width  of  an  ERC  line  indicates  the  significance  of  the 
covariance  between  two  waveforms,  with  the  scale  appearing  above  the  word  “significance.”  The 
color  of  the  line  indicates  the  time  delay  in  msec  (lag  time  of  maximum  covariance)  as  shown  in 
the  scale  above  “msec  delay.”  The  color  of  the  arrow  indicates  the  sign  of  the  covariance  (same 
color  as  line  =  positive;  skin  color  =  negative).  The  arrow  points  from  leading  to  lagging  channel, 
unless  there  is  no  delay,  in  which  case  a  bar  is  shown.  The  covariance  between  aPz  and  aCz  is 
significant  at  p  <  10“’.  The  aPz  waveform  leads  the  aCz  waveform  by  about  16-31  msec  (green 
line),  and  the  covariance  is  positive  (green  arrow)  (From  Gevins  et  at  ”) 
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8.  The  significance  of  ERCs  is  detennined  by  reference  to  an  estimate  of  the 
standard  deviation  of  the  “noise"  ERC.  The  noise  ERC  is  computed  by  averag¬ 
ing  random  intervals  in  each  single  trial  of  the  ensemble  of  trials.  ERC  analysis 
is  then  performed  on  a  filtered  and  decimated  version  of  the  resulting  “noise” 
averages,  yielding  a  distribution  of  “noise”  ERCs.  The  threshold  for  significance 
is  reduced  according  to  the  dimensionality  of  the  data  with  Duncan’s  correction 
procedure.  The  number  of  channels  is  used  as  a  conservative  estimate  of  the 
number  of  independent  dimensions.  The  most  significant  ERCs  in  each  interval 
are  graphed. 

9.  ANOVA  and  post-hoc  t  tests  are  used  to  compare  ERC  patterns  between 
conditions.  The  similarity  of  appearance  of  two  ERC  graphs  is  measured  with 
an  estimate  of  the  correlation  between  them.  The  estimate  comes  from  a 
distribution-independent  “bootstrap”  Monte  Carlo  procedure,'^  which  also 
yields  a  confidence  interval  for  the  estimates. 

10.  The  between-subject  variability  of  ERC  patterns  is  tested  by  determining 
whether  each  pair  of  experimental  conditions  of  a  particular  subject  can  be 
distinguished  using  discriminating  equations  generated  on  the  other  subjects. 

11.  The  within-subject  reliability  is  assessed  by  attempting  to  discriminate  the 
experimental  conditions  for  each  session  using  equations  generated  on  that 
subject's  other  sessions. 

The  tests  of  both  between-  and  within-subject  variability  and  reliability  are  per¬ 
formed  on  sets  of  single  trials.  This  quantifies  the  extent  to  which  the  condition-specific 
patterns  from  the  ERC  analysis  of  the  average  ERPs  can  be  observed  in  each  trial. 
Although  this  procedure  could  be  done  with  any  type  of  discriminant  analysis,  we  have 
developed  the  use  of  distribution-independent,  layered,  artificial  “neural  network” 
pattern  classification  algorithms  for  this  purpose.'^  '*  We  have  shown  that  this  method 
has  better  sensitivity  than  stepwise  or  full-model  linear  or  quadratic  discriminant 
analysis.’’  The  pattern  recognition  approach  has  the  advantage  of  testing  how  well  a 
subject’s  individual  trials  conform  to  those  of  the  group  in  discriminating  two  behav¬ 
ioral  conditions  of  interest.  In  the  same  way,  the  trials  of  each  session  of  a  subject 
are  tested  by  conformity  to  trials  from  the  other  sessions  of  that  subject.  Requiring 
trial-by-trial  discriminability  is  a  strict  condition  for  deciding  between-subject  variabil¬ 
ity  and  within-subject  reliability. 

Each  subject’s  classification  yields  a  score,  which  is  the  percent  of  trials  that  are 
correctly  clarified  by  the  group  discrimination  equations.  The  score  is  assessed  for 
significance  by  comparison  to  the  binomial  distribution.”  A  significant  classification 
score  for  a  subject  indicates  that  the  group  equations  are  successful  in  discriminating 
the  two  conditions  in  his  or  her  trials. 

Within-subject  (between-session)  reliability  is  tested  in  a  similar  manner.  The  trial 
set  (consisting  of  the  two  conditions)  from  each  of  a  subject’s  sessions  is  tested  with 
equations  developed  on  the  trial  sets  from  his  or  her  other  sessions.  The  single-trial 
ERC  values  come  from  channel  pairs  that  are  significant  in  the  ERC  pattern  formed 
from  the  average  over  all  his  or  her  sessions.  Post-hoc  comparisons  are  valuable  in 
determining  whether  effects  of  learning  and/or  habituation  are  evident  over  sessions, 
by  indicating  which  sessions  are  alike  and  where  transitions  occur  between  sessions. 

Figure  7  is  a  block  diagram  of  the  data  collection  and  analysis  process  discussed 
above.  For  studies  not  requiring  pattern  recognition  analysis,  the  event-related  covari¬ 
ances  are  computed  on  averaged  event-related  potentials  after  band-pass  filtering. 

In  the  next  section,  results  of  a  study  of  bimanual  visuomotor  performance  and  a 
study  of  the  effects  of  mental  fatigue  on  human  cognitive  networks  are  presented. 
Preliminary  results  of  a  study  of  elementary  language  processes  are  also  described. 
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AOIEEG  V  NEUROCOGNITIVE 
PATTERN  RECOGNITION 
SYSTEM 


UMOrtlMa 


FIGURE  7.  ADIEEG-V  system  for  pattern  recognition  of  event-related  brain  signals.  Separate 
subsystems  perform  on-line  experimental  control  and  data  collection,  data  selection  and  equa¬ 
tion,  signal  processing  and  pattern  recognition.  Current  capacity  is  128  channels. 
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APPLYING  THE  TOOLS 


Bimanual  Visuomotor  Task 


One  of  the  goals  of  the  bimanual  visuomotor  experiment  was  to  study  prefrontal 
involvement  while  subjects  prepared  to  perform  a  task  accurately  and  used  feedback 
about  their  accuracy  to  gauge  their  responses. 


Subjects  and  Task 


Seven  healthy,  right-handed,  male  adults  participated  in  this  study.  A  visual  cue. 
slanted  to  the  right  or  to  the  left,  prompted  the  subject  to  prepare  to  make  a  response 
pressure  with  the  right  or  left  index  finger.  One  sec  later,  the  cue  was  followed  by  a 
visual  numeric  stimulus  (numbers  1  -9)  indicating  that  a  pressure  of  0. 1  to  0.9  kg  should 
be  made  with  the  index  finger  of  the  previously  indicated  hand.  A  two-digit  number, 
presented  1  sec  after  the  peak  of  the  response  pressure,  provided  feedback  that  indicated 
the  subject’s  exact  pressure.  On  a  random  20%  of  the  trials,  the  stimulus  number  was 
slanted  in  the  opposite  direction  to  the  cue;  subjects  were  to  withhold  their  responses 
on  these  “catch  trials.”  The  next  trial  followed  1  sec  after  disappearance  of  the  feedback. 
Each  subject  performed  several  hundred  trials,  with  rest  breaks  as  needed. 


Recordings 


Twenty-six  channels  of  EEC  data,  as  well  as  vertical  and  horizontal  eye-movements 
and  flexor  digiton  muscle  activity  from  both  arms,  were  recorded.  All  single-trial 
EEC  data  were  screened  for  eye-movement,  muscle  potential,  and  other  artifacts,  and 
contaminated  data  were  discarded. 


Analysis  and  Results 


Intervals  used  for  ERC  analysis  were  centered  on  major  event-related  potential 
peaks.  ERCs  were  computed  between  each  of  the  120  pairwise  combinations  of  the  16 
nonperipheral  channels.  Intervals  were  set  from  5(X)  msec  before  the  cue  to  500  msec 
after  the  feedback.  We  first  calculated  the  mean  error  (deviation  from  the  required 
finger  pressure)  over  all  trials  from  the  recording  session.  Individual  trials  were  then 
classified  as  accurate  (trial  error  less  than  mean  error)  or  inaccurate  (trial  error  greater 
than  mean  error). 

ERC  patterns  during  a  37S-msec  interval,  centered  687  msec  post-cue  (spanning 
the  late  contingent  negative  variation— CNV),  regardless  of  subsequent  accuracy,  in¬ 
volved  left  prefrontal  sites,  as  well  as  appropnately  laterahzed  central  and  parietal  sites 
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(Fig.  8A  and  B).  Inaccurate  performance  by  the  right  hand  was  preceded  by  a  very 
simple  pattern,  while  inaccurate  performance  by  the  left  hand  was  preceded  by  a 
complex,  spatially  diffuse  pattern.  The  relative  lack  of  ERCs  preceding  inaccurate 
right-hand  performance  may  simply  reflect  inattention  on  those  trials,  while  the  strong 
and  complex  patterns  preceding  inaccurate  performance  with  the  left  hand  may  reflect 
effortful,  but  inappropriate,  preparation  by  the  right-handed  subjects. 

ERC  patterns  related  to  feedback  about  accurate  and  inaccurate  performances  were 
similar  immediately  after  the  onset  of  feedback,  but  began  to  differ  in  an  interval, 
centered  at  37S  msec,  that  spanned  the  P3  (Fig.  9A  and  B).  The  ERC  patterns  for 
feedback  to  accurate  performance  by  the  two  hands  were  very  similar  (bootstrap 
correlation  =  0.91  ±  0.01),  involving  midline  anterocentral,  central,  anteroparietal, 
parietal,  and  anterooccipital  sites;  left  anteroparietal  and  anterocentral  sites;  and  right 
parietal,  anteroparietal,  anterocentral,  and  frontal  sites.  These  accurate  patterns  in¬ 
volved  many  long-delay  (32-79  msec)  ERCs.  The  waveforms  of  the  frontal  and  antero¬ 
central  sites  lagged  those  of  more  posterior  sites.  For  feedback  to  inaccurate  perform¬ 
ance,  patterns  for  both  hands  were  also  very  similar  (bootstrap  correlation  =  0.90  ± 
0.02)  and  involved  most  of  the  same  sites  as  the  accurate  patterns,  with  the  striking 
inclusion  of  the  left  and  midline  frontal  sites.  Again,  front^  waveforms  lagged  those 
of  the  more  posterior  sites  with  which  they  covaried.  There  were  even  more  long-delay 
ERCs  than  in  the  accurate  patterns. 


Summary 


The  pre-stimulus  ERC  patterns  seem  to  characterize  a  distributed  preparatory 
neural  set  that  is  related  to  the  accuracy  of  subsequent  task  performance.  This  network 
involves  distinctive  cognitive  (frontal),  integrative-motor  (midline  precentral)  and  later- 
alized  somesthetic-motor  (central  and  parietal)  components.  The  involvement  of  the 
left-frontal  site  is  consistent  with  Teuta^’s”  notions  of  corollary  discharge  and  with 
other  experimental  and  clinical  findings  suggesting  the  synthesis  and  integration  of 
function^  networks  in  prefrontal  cortical  areas.  A  midline  anterocentral 
integrative-motor  component  is  consistent  with  known  involvement  of  premotor  and 
supplementary  motor  areas  in  initiating  motor  responses.  The  finding  of  an  appropri¬ 
ately  lateralized  central  and  parietal  component  is  consistent  with  evidence  from  pri¬ 
mates  and  humans  for  neuronal  firing  in  motor  and  somatosensory  cortices  prior  to 
motor  responses. 


FIGURE  >.  Preparatory  event-related  covariance  (ERC)  patterns  {colored  lines).  Measurements 
are  from  an  interval  300  to  873  msec  after  the  cue  for  subsequently  accurate  (A)  right-hand  and 
(B)  left-hand  visuomotor  task  performance  by  seven  right-handed  men.  The  ERCs  are  superim¬ 
posed  for  illustrative  purposes  over  a  horizontal  MR  scan.  The  thickness  of  a  covariance  line  is 
proportional  to  its  signiiirance  (from  .03  to  .003).  A  violet  line  indicates  the  covariance  is  positive, 
while  a  blue  line  is  negative.  ERCs  involving  left  frontal  and  appropriately  contralateral  central 
and  parietal  electrode  sites  are  prominent  in  patterns  for  subsequently  accurate  performance  of 
both  hands.  The  magnitude  and  number  of  preparatory  ERCs  are  greater  preceding  subsequently 
inaccurate  left-hand  performance  than  those  preceding  inaccurate  right-hand  performance  by 
the  right-handed  subjects.  Inaccurate  left-hand  preparatory  ERCs  are  more  widely  distributed 
compared  with  the  left-hand  accurate  pattern.  For  the  right-hand,  fewer  and  speaker  ERCs 
characterize  subsequently  inaccurate  performance. 
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Since  ERC  feedback  patterns  of  accurate  or  inaccurate  performance  (involving 
either  hand)  were  more  similar  than  those  between  accurate  and  inaccurate  patterns 
for  one  hand,  it  may  be  inferred  that  the  feedback  patterns  were  related  more  to 
performance  accuracy  than  to  the  hand  used.  The  fact  that  ERC  patterns  following 
disconfirming  feedback  involved  more  frontal  sites  than  did  patterns  following  confirm¬ 
ing  feedback  is  consistent  with  the  idea  that  greater  resetting  of  performance-related 
neural  systems  is  required  following  disconfirming  feedback.  Likewise,  the  front  focus 
of  these  differences  is  consistent  with  the  importance  of  the  frontal  lobes  in  the  integra¬ 
tion  of  sensory  and  motor  activities.“-^’ 


Effects  of  Mental  Fatigue  on  Functional  Brain  Topography 


In  this  study,  the  effects  of  mental  fatigue  on  preparation  and  memory,  stimulus 
recognition  and  stimulus  processing  were  studied.^'* 


Subjects  and  Task 


Five  healthy,  right-handed,  male  subjects  performed  a  task  that  required  that  they 
remember  two  continuously  changing  numbers,  in  the  presence  of  numeric  distractors, 
and  produce  precise  finger  pressures.  Each  trial  consist^  of  a  warning  symbol,  followed 
by  a  single-digit  visual  stimulus  to  be  remembered,  followed  by  the  subject’s  finger- 
pressure  response  to  the  stimulus  number  presented  two  trials  ago,  followed  by  a 
two-digit  feedback  number  indicating  the  accuracy  of  the  response.  For  example,  if  the 
stimulus  numbers  in  five  successive  trials  were  8, 6,  1,  9, 4,  the  correct  response  would 
be  a  pressure  of  0.8  kg  when  seeing  the  1,  0.6  kg  for  the  9,  and  0.1  kg  for  the  4.  To 
increase  the  task  difficulty,  subjects  were  required  to  withhold  their  response  on  a 
random  20%  of  the  trials.  These  “no-response  catch  trials”  were  trials  in  which  the 
current  stimulus  number  was  identical  to  the  stimulus  two  trials  ago.  Subjects  were 
given  ample  practice  to  stabilize  accuracy  and  reaction  time. 

Subjects  performed  the  task  over  a  10-14  h  period.  Sets  of  trials  with  equally 
accurate  performance  and  response  movement  parameters  were  selected  from  three 
periods:  an  early  period  during  the  first  7  h  (Alert),  a  middle  period  just  prior  to  any 
decline  in  over^  performance  accuracy  (Incipient  Performance  Impairment),  and  a 
late  period  after  performance  had  significantly  degraded. 


FIGURE  9.  Most  significant  (top  2  SD)  feedback  ERC  patterns  dieted  when  subjects  were  given 
information  about  the  accuracy  of  their  finger  pressure  response  by  the  right  hand  (A)  and  left 
hand  (B).  ERCs  were  derived  from  a  187-msec-wide  interval,  beginning  281  msec  and  ending  468 
msec  post-feedback,  on  theta-band-filtered,  7-subject-averaged  evoked  potential  waveforms.  The 
color  of  the  covariance  line  indicates  the  lag  time  of  maximum  covariance  between  electrodes: 
yellow,  0-15  msec;  green,  16-31  msec;  blue,  32-47  msec;  red,  48-79  msec;  purple,  80-t-  msec.  A 
major  difference  between  accurate  and  inaccurate  patterns  is  that  the  left  and  midline  frontal  sites 
are  only  involved  in  the  inaccurate  patterns.  The  involvement  of  these  sites  may  reflect  greater 
processing  after  inaccurate  performance  in  order  to  improve  subsequent  performance. 
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Recordings 


EEGs  were  recorded  with  either  33  or  51  channels  set  in  a  nylon  mesh  cap.  Vertical 
and  horizontal  eye  movements  were  also  recorded,  as  were  the  responding  flexor  digitori 
muscle  potentials,  electrocardiogram,  and  respiration.  Three-axis  magnetic  resonance 
image  scans  were  made  of  three  of  the  five  subjects. 


Analysis  and  Results 


Neuroelectric  effects  were  observed  during  two  fraction-of-a-second  intervals  when 
subjects  (1)  prepared  to  receive  a  new  stimulus  number  while  holding  the  two  previous 
stimulus  numbers  in  working  memory  (CNV  interval),  and  (2)  when  they  withheld 
their  response  in  the  instance  where  the  current  stimulus  number  was  the  same  as  the 
two-back  stimulus  number  (P300  intervai).  Significant  differences  were  seen  between 
the  early  Alert  period  and  the  middle  Incipient  Performance  Impairment  (IPI)  period 
(p  <  0.0001).  While  the  magnitude  of  the  patterns  was  reduced  during  both  preparatory 
and  response  inhibition  intervals,  the  topographic  distribution  of  the  pattern  was  only 
affected  during  preparation  (Fig.  10).  The  preparatory  pattern  shifted  from  one  strongly 
focused  on  midline  central  and  precentral  sites  to  one  focused  primarily  on  right-sided 
precentral  and  parietal  sites.  It  appeared  as  though  extended  task-performance  altered 
the  “neural  strategy”  used  to  perform  the  same  behavior.  Thus,  prolonged  mental  work 
differentially  affected  two  successive  split-second  information  processing  intervals. 

The  extent  to  which  each  subject’s  patterns  corresponded  to  the  group’s  and  the 
extent  to  which  it  was  possible  to  distinguish  individual  trials  from  the  Alert  and  IPI 
periods  of  the  session  were  determined  next  using  pattern  recognition  analysis.  ERCs 
common  to  the  group  (Fic.  10,  left  and  middle)  were  considered  as  possible  variables. 
For  the  preparatory  interval,  they  consisted  of  ERCs  computed  over  the  500  msec 
pre-stimulus  epoch  of  each  trial.  For  separate  groups  of  three  and  two  subjects  whose 
resting  EEG  characteristics  differed,  five  equations  were  formed  on  four-fifths  of  the 
trials,  and  tested  on  the  remaining  one-fifth.  The  average  test  set  accuracy  of  Alert 
versus  IPI  discrimination  was  then  computed  and  tested  for  significance  by  reference 
to  the  binomial  distribution.  Discrimination  accuracy  was  62%  (p  <  0.001).  Individual¬ 
ized  equations  were  generated  on  the  subject  with  the  most  usable  data,  still  using  the 
variables  from  the  group  pattern.  Discrimination  accuracy  climbed  to  81%  (p  < 
0.0001). 


Summary 


Striking  changes  occurred  in  the  ERC  patterns  after  subjects  performed  the  difficult 
memory  and  fine-motor  control  task  for  an  average  of  7-9  h,  but  before  performance 
deteriorated.  Pattern  strength  was  reduced  in  a  fraction-of-a-second-long  response 
preparation  interval  over  midline  precentral  areas  and  over  the  entire  left  hemisphere. 
By  contrast,  pattern  strength  in  a  succeeding  response-inhibition  interval  was  reduced 
over  all  areas.  The  pattern  changed  least  in  an  intervening  interval  associated  with 
visual-stimulus  processing.  This  suggests  that,  in  addition  to  the  well-known  global 
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reduction  in  neuroelectric  signal  strength,  functional  neural  networks  are  selectively 
affected  by  sustained  mental  work  in  specific  fraction-of-a-second  task  intervals.  For 
practical  application,  these  results  demonstrate  the  possibility  of  detecting  leading 
indicator  neuroelectric  patterns  which  precede  degradation  of  performance  due  to 
sustained  mental  work. 


Neurocognitive  Analysis  of  Elementary  Language  Processes 


Preliminary  results  of  a  recent  experiment  demonstrate  good  spatial  and  temporal 
differentiation  of  basic  linguistic  functions  using  S9-channel  EEG  recordings. 


Subjects  and  Task 


Nine  right-handed,  healthy  male  subjects  performed  a  language  task  in  which  they 
had  to  judge  whether  the  second  visually  presented  stimulus  of  a  given  condition  (S2) 
formed  a  match  with  the  first  stimulus  (SI).  There  were  four,  fully  randomized  condi¬ 
tions  in  the  experiment.  In  the  graphic  non-letter  condition,  the  stimuli  were  characters 
of  Katakana,  a  Japanese  script  with  which  none  of  the  subjects  were  familiar.  Subjects 
were  required  to  judge  whether  or  not  the  stimuli  were  identical.  In  the  phonemic 
condition,  subjects  were  required  to  judge  if  the  pronounceable  but  neologistic  word 
stimuli  sounded  alike.  In  the  semantic  condition,  subjects  were  required  to  decide  if 
the  high  frequency,  open-class  monosyllabic  words  were  opposite  or  not.  Finally,  in 
the  grammatical  condition,  subjects  judged  if  the  verb  of  the  S2  formed  a  meaningful 
and  grammatically  correct  sentence  with  the  SI  pronoun.  Eighty-five  percent  of  the 
trials  were  “match”  trials  and  no  response  was  required;  subjects  responded  to  the 
15%  mismatch  trials  with  a  button  press  using  the  left  index  finger. 


Recordings 


EEGs  referenced  to  the  midline  anterior-parietal  (aPz)  electrode  were  recorded 
from  59  scalp  electrodes.  The  montage  was  an  extended  10-20  system,^  and  included 
the  frontal  sites  aFl  and  aF2,  Fz,  F3  to  F8,  and  Fpz;  the  anterior  central  aCz,  aCl  to 
aC6;  central  Cz,  C3  and  C4;  anterior  temporal  aT5  and  aT6;  temporal  T3  to  T8;  lower 
temporal  ITl,  1T2,  ITS,  1T6;  ventral  temporal  vT5  and  vT6;  anterior  parietal  aPl  to 
aP6;  parietal  Pz,  P3  to  P6;  anterior  occipital  aOl  and  a02;  occipital  Oz,  01  and  02; 
ventral  occipital  vOl  and  v02;  and  the  inion  (I)  and  both  mastoids  (Ml  and  M2). 
Vertical  eye  movements  were  recorded  bipolarly  from  an  electrode  pair  placed  supra- 
and  suborbitally;  horizontal  eye  movements  were  recorded  bipolarly  tetween  electrodes 
at  the  outer  canthus  of  each  eye.  Other  bipolar  pairs  were  placed  over  flexor  digitori 
muscles  of  left  and  right  arms  to  record  EMG  and  at  the  submentalis  to  record  subvocal 
movements  of  the  larynx  and  mouth.  The  EEG  was  amplified  8333  times,  band-pass 
filtered  from  0.5-50  Hz  and  recorded  at  128  samples  per  sec. 


FIGUKE  10.  Pattern  recognition  analysis  using  an  artificial,  layered  neural  network  that  distin- 
guished  ERC  neuroelectric  patterns  recorded  during  baseline  (early),  incipient  performance  im¬ 
pairment  (middle),  and  impaired  performance  (late)  periods  from  five  Air  Force  test  pilots  per¬ 
forming  a  difficult  visuomotor-memory  task  over  a  14-h  period.  Baseline  data  were  obtained 
during  the  first  7  hours;  incipient  performance  impairment  data  during  hours  7- 10  preceding 
impaired  performance;  the  impaired  performance  data  were  obtained  during  hours  10- 14.  The 
ERCs  were  measured  during  a  SOO-msec  interval  when  the  subjects  were  remembering  two 
numbers  and  preparing  for  the  next  stimulus.  ERCs  greatly  declin^  in  magnitude  from  baseline 
(early)  to  incipient  performance  impairment  (middle)  to  impaired  performance  (late)  epochs.  The 
patterns  also  changed,  with  the  emphasis  shifting  from  the  (1)  midline  central,  (2)  midline 
precentral,  and  (3)  left  parietal  sites  to  right  hemisphere  sites. 


SYNTACTIC  VS  SEMANTIC 
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FIGURE  11.  LD  waveforms  evoked  by  the  first  of  two  stimuli  (syntactic  and  semantic)  in  an 
experiment  designed  to  study  elementary  language  processes.  The  waveforms  are  averaged  across 
nine  subjects.  The  major  difference  between  the  grammatic  (syntactic)  and  semantic  conditions  is 
lateralized  to  the  left  hemisphere,  where  the  grammatic  conation  has  a  substantial  peak  at  442 
msec  at  left  frontal  and  anterior  central  sites.  The  x-axis  shows  1-sec  beginning  with  the  SI.  The 
y-axis  corresponds  to  ±  0.153  ;iV/sq  cm. 
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Analysis  and  Results 


Analysts  of  both  ERP  and  ERC  data  is  ongoing.  Among  the  condition  differences 
observed  to  date,  for  example,  syntactic  and  nonsyntactic  trials  were  clearly  differenti¬ 
ated  by  ERP  topography.  After  SI,  the  grammatic  (syntactic)  condition  alone  had  a 
substantial  peak  at  442  msec  at  lateral  frontal  and  anterior  central  sites.  N442  was 
most  robust  at  F3  (Fig.  1 1),  FS,  and  aCl,  where  it  was  significantly  larger  than  in  the 
semantic  condition  (p  <  O.OS).  After  S2  (not  shown),  the  grammatic  condition  was 
again  distinguished  from  the  semantic  condition  by  a  positive  peak  at  279  msec  at  left 
frontal  and  anterior  central  electrodes.  At  these  sites,  the  grammatic  P279  was  larger 
in  amplitude  than  in  the  semantic  condition  (p  <  O.OS).  It  was  not  significantly 
lateralized. 


Summary 


Stimuli  in  both  the  nonsyntactic  and  syntactic  conditions  in  this  experiment  were 
words  with  similar  physical  characteristics.  The  main  difference  between  them  post-Sl 
was  that  the  semantic  (nonsyntactic)  condition  used  open-class  words  (content  words 
such  as  nouns  and  adjectives)  and  the  grammatic  (syntactic)  condition  used  closed-class 
words  (fimction  words),  in  this  case  pronouns.  It  is  possible  that  the  N41 1  observed 
for  the  syntactic  con<htion  is  related  to  processing  the  closed-class  words,  to  the 
initiation  of  a  “syntactic  parser,”^’  or  to  both  processes  simultaneously.  The  location 
of  the  “syntactic”  effect  at  left  frontal  sites  0F^3,  F3,  and  aC3)  after  both  SI  and  S2  is 
consistent  with  neurophysiological  observations  of  syntactic  deficits  and  difiSculties  in 
handling  closed-class  words  in  aphasia  patients  whose  lesions  involve  and  extend  deep 
to  Broca’s  area. 


CONCLUSIONS 

Methodological 


Advances  in  neuroelectric  recording  and  analysis  technology  during  the  past  two 
decades  have  significantly  increased  the  sensitivity  and  specificity  of  measuring  brain- 
behaviOT  eeiatkniships.  Sharper  spatial  resolution  is  provided  by  an  increased  number 
of  electrodes  and  by  use  of  the  Lsplacian  derivation.  Information  about  common 
activity  and  its  temporal  relationships  is  provided  by  the  event-related  covariance 
measure,  while  neural  network  pattern  recognition  analysis  provides  a  powerful  method 
of  detecting  neurocognitive  signals  in  sets  of  single-trial  data.  The  signs  we  have 
seen  of  rapidly  shifting,  functionally  interdependent  cortical  networks  are  particularly 
intriguing  in  Ught  of  their  consistency  with  both  historical  and  contemporary,  clinicd 
and  experimental  findings  about  brain-behavior  relationships.  In  our  current  research, 
we  hope  to  further  refine  and  elaborate  these  methodological  and  experimental  para¬ 
digms. 
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Models  of  Neural  Information  Processing  in  Cognitive  Electrophysiology 


Because  of  the  stimulus-response  design  inherent  in  most  experimental  designs, 
many  models  of  cognitive  functioning  have  a  passive  tone.  The  brain  reacts  to  a  given 
stimulus,  and  the  stages  leading  to  response  are  inferred  from  measures  of  reaction 
time,  ERP  peak  latencies,  and  so  on.  However,  we  know  from  experience,  observation, 
and  inference  that  cognitive  processes  are  highly  interactive.  Our  environment  is,  in  a 
sense,  altered  by  our  perception  of  it,  because  perception  itself  is  a  synthesis  of  sensation, 
current  brain  state,  and  past  cognitive  experience.  This  synthesis  involves  a  continu¬ 
ously  updated,  dynamic  internal  representation  of  what  we  imagine  our  selves  and 


FIGURE  12.  Sketch  of  parallel,  sequential,  hierarchically  organized  information  processing  in 
functional  networks  of  the  human  neocortex.  The  previous  moment’s  internal  model  influences 
the  current  moment's  goal  direction  and  attention,  which  in  turn  influences  other  stages  of 
processing. 


enviromiMiit  to  be  like  at  any  given  moment  Moreover,  we  use  our  effector  and  sensory 
systems  to  probe  actively  the  environment  for  information  relevant  to  maintaining  and 
updating  the  self/world  model  (Fic.  12).  Each  perception,  each  action  is  incorporated 
into  the  internal  model,  and  new  perceptions  and  actions  are  in  turn  influenced  through 
the  model’s  role  in  directing  attentional  and  conceptual  processes.  It  is  challenging, 
but  not  impossible,  to  design  experimental  situations  which  emphasize  this  dynamic 
and  interactive  nature  of  cognition.  Two  areas  that  have  been  of  particular  interest  to 
us  are  preparatory  processes,  which  precede  the  stimulus  and  are  directed  by  the 
internal  m^el,  and  feedback,  which  governs  the  updating  of  the  model  after  behaviors. 
Although  it  is  likely  that  the  frontal  cortex  plays  a  pivotal  role  in  both  processes,  it 
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would  be  simplistic  to  consider  the  frontal  lobes  as  a  mere  executor.  Rather,  it  is  likely 
that  the  entire  brain  is  involved  in  a  constellation  of  rapidly  changing,  function^ 
networks  that  provide  the  delicate  balance  between  stimulus-locked  behavior  and 
purely  imaginary  ideation.  The  pre-stimulus  and  the  feedback-associated  “processing 
networks”  observed  in  our  studies  may  be  signs  of  such  interrelated  activity.  With  even 
further  advances  in  brain  imaging  in  neuropsychophysiology  in  the  1990s,  we  can  hope 
to  achieve  increasingly  detailed  and  direct  measurements  of  the  organization  and 
interrelationships  of  sensory  and  higher  cognitive  behaviors  in  health  and  in  disease. 
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